Exact and approximate strategies for symmetry reduction in model checking

Symmetry reduction techniques can help to combat the state space explosion problem for model checking, but are restricted by the hard problem of determining equivalence of states during search. Consequently, existing symmetry reduction packages can only exploit full symmetry between system components, as checking the equivalence of states is straightforward in this special case. We present a framework for symmetry reduction with an arbitrary group of structural symmetries. By generalising existing techniques for efficiently exploiting symmetry, and introducing an approximate strategy for use with groups for which fast, exact strategies are not available, our approach allows for significant state-space reduction with minimal time overhead. We show how computational group theoretic techniques can be used to analyse the structure of a symmetry group so that an appropriate symmetry reduction strategy can be chosen, and we describe a symmetry reduction package for the Spin model checker which interfaces with the computational algebra system Gap. Experimental results on a variety of Promela models illustrate the effectiveness of our methods

Automating {UML} Models Merge for Web Services Testing

International audienceThis paper presents a method for merging UML models which takes place in a quality evaluation framework for Web Services (WS). This framework, called iTac-QoS, is an ex- tended UDDI server (a yellow pages system dedicated to WS), using model based testing to assess quality. WS ven- dors have to create UML model of their product and our framework extracts tests from it. Depending on the results of the test execution, a mark is given to WS. This mark per- mits to customers to have an idea about the quality of WS they find on our UDDI server. Up today, our framework was limited to WS which did not use other WS. This was justified by the fact that it is im- possible for vendors to create a good model of a foreign product. Our method for model merging solves this prob- lem: each vendor produces models of its own product, and we automatically merge the different models. The resulting model from this merging represents the composition of the different WS. For each type of diagram present in the models (class, instance or state-chart diagram), a method is proposed in order to produce a unique model. In addition to this, a solu- tion is proposed to merge all OCL code in the class modeling the WS under test. Unfortunately, this process introduces inconsistencies in the resulting model, that falsify the results of the subsequent test generation phase. We thus propose to detect such inconsistencies in order to distinguish incon- sistent and unreachable test targets

A Constraint Solver for PHP Arrays

International audienceIn previous works, we have proposed Praspel, a framework for contract-based testing in PHP. Among others, it includes a specification language and a unit test generator which automatically generates test data from formal preconditions. The generator sometimes rejects data, when they do not satisfy parts of the preconditions. In many cases, generation with rejection is not efficient enough. Thus we investigate practical contexts where more efficient generation algorithms can be designed and we extend Praspel with their implementation. After strings, that we have already considered, the most frequent data type in PHP is arrays. They cover most of the needs for collections, because they can store key-value pairs of any kind, they do not have a specific length or depth, and they are efficiently implemented. In this paper, we report on a study to know what are the most popular constraints on PHP arrays. Then we formalize these constraints and we present an implementation in PHP of a constraint solver for these constraints. In this context, the constraint-based approach removes all the rejections

Extending the Unified Process with Model-Based Testing

International audienceThe Unified Process (UP) is a software development tech- nique that includes modeling of specifications and testing workflow. This workflow is achieved by information interpretation of specification to pro- duce manual tests. In this paper, we extend the UP with model-based testing (MBT) where models resulting of the UP will be used for MBT. We describe how model-based testing introduces new test design activi- ties in parallel with the application design activities. We give guidelines to derive the test model from the analysis model produced by the UP. We illustrate this tailored process with the example of a Geneva State taxation

Safety Property Driven Test Generation from {JML} Specifications

International audienceThis paper describes the automated generation of test sequences derived from a JML specification and a safety property written in an ad hoc language, named JTPL. The functional JML model is animated to build the test sequences w.r.t. the safety properties, which represent the test targets. From these properties, we derive strategies that are used to guide the symbolic animation. Moreover, additional JML annotations reinforce the oracle in order to guarantee that the safety properties are not violated during the execution of the test suite. Finally, we illustrate this approach on an industrial JavaCard case study

Praspel: Contract-Driven Testing for PHP using Realistic Domains

We present an integrated contract-based testing framework for PHP. It relies on a behavioral interface specification language called Praspel, for "PHP Realistic Annotation and Specification Language". Using Praspel developers can easily annotate their PHP scripts with formal contracts, namely class invariants, and method pre- and postconditions. These contracts describe assertions either by predicates or by assigning realistic domains to data. Realistic domains introduce types in PHP and describe complex structures frequently encountered in applications, such as email addresses or SQL queries. Realistic domains display two properties: predicability, which allows to check if a data belongs to a given realistic domain, and samplability, which allows to generate valid data. This paper introduces coverage criteria dedicated to contracts, designed to exhibit relevant behaviors of the annotated methods. Test data are then computed to satisfy these coverage criteria, by using dedicated data generators for complex realistic domains, such as arrays or strings. This framework has been implemented and disseminated within the PHP community, which gave us feedback on their usage of the tool and the relevance of this integrated process with respect to their practice of manual testing

Associer des techniques de preuve et de résolution de contraintes pour la construction d'abstractions

National audienceCet article présente une méthode de génération assistée de tests. Elle applique des critères dynamiques de sélection des tests (TP) sur un modèle formel comportemental (M) utilisé auparavant, par exemple par LTG, pour générer des tests fonctionnels à partir de critères statiques de sélection. On peut appliquer à M un critère dynamique de sélection TP mais ceci nécessite de représenterM par un automate. Pour des applications réelles, sa taille en nombre d'états et de transitions est beaucoup trop grande (voir infinie) pour être utilisable. Nous proposons une méthode pour extraire une abstraction de M à partir d'un objectif de test TP. Nous effectuons un produit synchronisé de cette abstraction avec TP afin de cibler les exécutions du système sous test qui satisfont TP. Puis nous générons des tests abstraits symboliques à partir de ce modèle réduit en appliquant les critères de couverture tous les états ou toutes les transitions. Cet ensemble de tests est valué à partir de M, concrétisé puis exécuté sur l'implémentation sous test. Cette méthode est proposée pour compléter la méthode BZ-TT de génération de tests à partir de critères statiques de sélection. L'utilisateur obtient des tests complémentaires en fournissant un critère dynamique de sélection. La méthode réutilise M, la couche de concrétisation des tests et l'infrastructure d'exécution des tests. L'originalité de l'approche est de construire une abstraction du modèle issue automatiquement de l'analyse statique d'un objectif de test formalisant des besoins de test d'une propriété dynamique du système

Génération de tests à partir de critères dynamiques de sélection et par abstraction

International audienceCet article présente une méthode de génération assistée de tests. Elle applique des critères dynamiques de sélection des tests (TP) sur un modèle formel comportemental (M) utilisé auparavant, par exemple par LTG, pour générer des tests fonctionnels à partir de critères statiques de sélection. On peut appliquer à M un critère dynamique de sélection TP mais ceci nécessite de représenter M par un automate. Pour des applications réelles, sa taille en nombre d'états et de transitions est beaucoup trop grande (voir infinie) pour être utilisable. Nous proposons une méthode pour extraire une abstraction de M à partir d'un objectif de test TP. Nous effectuons un produit synchronisé de cette abstraction avec TP afin de cibler les exécutions du système sous test qui satisfont TP. Puis nous générons des tests abstraits symboliques à partir de ce modèle réduit en appliquant les critères de couverture tous les états ou toutes les transitions. Cet ensemble de tests est valué à partir de M, concrétisé puis exécuté sur l'implémentation sous test. Cette méthode est proposée pour compléter la méthode BZ-TT de génération de tests à partir de critères statiques de sélection. L'utilisateur obtient des tests complémentaires en fournissant un critère dynamique de sélection. La méthode réutilise M, la couche de concrétisation des tests et l'infrastructure d'exécution des tests. L'originalité de l'approche est de construire une abstraction du modèle issue automatiquement de l'analyse statique d'un objectif de test formalisant des besoins de test d'une propriété dynamique du système

Automated UML models merging for web services testing

International audienceThis paper presents a method for merging UML models which takes place in a quality evaluation framework for Web Services (WS). This framework, called iTac-QoS, is an extended UDDI server (a yellow pages system dedicated to WS), using model based testing to assess quality. WS vendors have to create UML model of their product and our framework extracts tests from it. Depending on the results of the test execution, a mark is given to WS. This mark gives to the cus- tomers an idea about the quality of WS they find on our UDDI server. Up today, our framework was limited to WS which did not use other WS. This was justified by the fact that it is impossible for vendors to cre- ate a good model of a foreign product. Our method for model merging solves this problem: each vendor produces models of its own product, and we automatically merge the different models. The resulting model from this merging represents the composition of the different WS. For each type of diagram present in the models (class, instance or state- chart diagram), a method is proposed in order to produce a unique model. In addition to this, a solution is proposed to merge all OCL code in the class modeling the WS under test. Unfortunately, this pro- cess introduces inconsistencies in the resulting model, that falsify the results of the subsequent test generation phase. We thus propose to detect such inconsistencies in order to distinguish inconsistent and un- reachable test targets